A Provable Defense for Deep Residual Networks

We present a training system, which can provably defend significantly larger neural networks than previously possible, including ResNet-34 and DenseNet-100. Our approach is based on differentiable abstract interpretation and introduces two novel concepts: (i) abstract layers for fine-tuning the precision and scalability of the abstraction, (ii) a flexible domain specific language (DSL) for describing training objectives that combine abstract and concrete losses with arbitrary specifications. Our training method is implemented in the DiffAI system

On-line processing of English which-questions by children and adults: a visual world paradigm study

Previous research has shown that children demonstrate similar sentence processing reflexes to those observed in adults, but they have difficulties revising an erroneous initial interpretation when they process garden-path sentences, passives, and wh -questions. We used the visual-world paradigm to examine children's use of syntactic and non-syntactic information to resolve syntactic ambiguity by extending our understanding of number features as a cue for interpretation to which -subject and which -object questions. We compared children's and adults’ eye-movements to understand how this information shapes children's commitment to and revision of possible interpretations of these questions. The results showed that English-speaking adults and children both exhibit an initial preference to interpret an object- which question as a subject question. While adults quickly override this preference, children take significantly longer, showing an overall processing difficulty for object questions. Crucially, their recovery from an initially erroneous interpretation is speeded when disambiguating number agreement features are present

Privacy Risks of Securing Machine Learning Models against Adversarial Examples

The arms race between attacks and defenses for machine learning models has come to a forefront in recent years, in both the security community and the privacy community. However, one big limitation of previous research is that the security domain and the privacy domain have typically been considered separately. It is thus unclear whether the defense methods in one domain will have any unexpected impact on the other domain. In this paper, we take a step towards resolving this limitation by combining the two domains. In particular, we measure the success of membership inference attacks against six state-of-the-art defense methods that mitigate the risk of adversarial examples (i.e., evasion attacks). Membership inference attacks determine whether or not an individual data record has been part of a model's training set. The accuracy of such attacks reflects the information leakage of training algorithms about individual members of the training set. Adversarial defense methods against adversarial examples influence the model's decision boundaries such that model predictions remain unchanged for a small area around each input. However, this objective is optimized on training data. Thus, individual data records in the training set have a significant influence on robust models. This makes the models more vulnerable to inference attacks. To perform the membership inference attacks, we leverage the existing inference methods that exploit model predictions. We also propose two new inference methods that exploit structural properties of robust models on adversarially perturbed data. Our experimental evaluation demonstrates that compared with the natural training (undefended) approach, adversarial defense methods can indeed increase the target model's risk against membership inference attacks.Comment: ACM CCS 2019, code is available at https://github.com/inspire-group/privacy-vs-robustnes

ReluDiff: Differential Verification of Deep Neural Networks

As deep neural networks are increasingly being deployed in practice, their efficiency has become an important issue. While there are compression techniques for reducing the network's size, energy consumption and computational requirement, they only demonstrate empirically that there is no loss of accuracy, but lack formal guarantees of the compressed network, e.g., in the presence of adversarial examples. Existing verification techniques such as Reluplex, ReluVal, and DeepPoly provide formal guarantees, but they are designed for analyzing a single network instead of the relationship between two networks. To fill the gap, we develop a new method for differential verification of two closely related networks. Our method consists of a fast but approximate forward interval analysis pass followed by a backward pass that iteratively refines the approximation until the desired property is verified. We have two main innovations. During the forward pass, we exploit structural and behavioral similarities of the two networks to more accurately bound the difference between the output neurons of the two networks. Then in the backward pass, we leverage the gradient differences to more accurately compute the most beneficial refinement. Our experiments show that, compared to state-of-the-art verification tools, our method can achieve orders-of-magnitude speedup and prove many more properties than existing tools.Comment: Extended version of ICSE 2020 paper. This version includes an appendix with proofs for some of the content in section 4.

Assessing and mapping language, attention and executive multidimensional deficits in stroke aphasia.

There is growing awareness that aphasia following a stroke can include deficits in other cognitive functions and that these are predictive of certain aspects of language function, recovery and rehabilitation. However, data on attentional and executive (dys)functions in individuals with stroke aphasia are still scarce and the relationship to underlying lesions is rarely explored. Accordingly in this investigation, an extensive selection of standardized non-verbal neuropsychological tests was administered to 38 individuals with chronic post-stroke aphasia, in addition to detailed language testing and MRI. To establish the core components underlying the variable patients' performance, behavioural data were explored with rotated principal component analyses, first separately for the non-verbal and language tests, then in a combined analysis including all tests. Three orthogonal components for the non-verbal tests were extracted, which were interpreted as shift-update, inhibit-generate and speed. Three components were also extracted for the language tests, representing phonology, semantics and speech quanta. Individual continuous scores on each component were then included in a voxel-based correlational methodology analysis, yielding significant clusters for all components. The shift-update component was associated with a posterior left temporo-occipital and bilateral medial parietal cluster, the inhibit-generate component was mainly associated with left frontal and bilateral medial frontal regions, and the speed component with several small right-sided fronto-parieto-occipital clusters. Two complementary multivariate brain-behaviour mapping methods were also used, which showed converging results. Together the results suggest that a range of brain regions are involved in attention and executive functioning, and that these non-language domains play a role in the abilities of patients with chronic aphasia. In conclusion, our findings confirm and extend our understanding of the multidimensionality of stroke aphasia, emphasize the importance of assessing non-verbal cognition in this patient group and provide directions for future research and clinical practice. We also briefly compare and discuss univariate and multivariate methods for brain-behaviour mapping

Penilaian Kinerja Keuangan Koperasi di Kabupaten Pelalawan

This paper describe development and financial performance of cooperative in District Pelalawan among 2007 - 2008. Studies on primary and secondary cooperative in 12 sub-districts. Method in this stady use performance measuring of productivity, efficiency, growth, liquidity, and solvability of cooperative. Productivity of cooperative in Pelalawan was highly but efficiency still low. Profit and income were highly, even liquidity of cooperative very high, and solvability was good